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SmMARY 

This  study  considers  the  problem  of  determining  step  function 
maximum  likelihood  estimates  for  N  stochastically  ordered  distributions, 
subject  to  the  constraint  that  the  estimates  themselves  must  also  be 
stochastically  ordered.  This  problem  arises,  for  example,  in  the 
context  of  reliability  growth.  Brunk,  et  al.,  [2]  has  achieved  a 
closed  form  solution  for  the  case  N  *•  2,  but  was  unable  to  extend  the 
results  to  the  case  N  >  2.  We  shall  present  a  new  analytical  method 
based  on  the  Kuhn-Tucker  optimality  conditions  for  the  equivalent 
concave  program.  For  N  ■  2,  the  method  yields  the  closed  form  solution 
of  Brunk,  and  for  N  ^  3,  the  method  yields»when  used  in  conjunction 
with  a  reduction  strategy  developed  in  [4]»an  efficient  computational 
algorithm.  The  algorithm  involves  solving  a  short  sequence  of 
essentially  unconstrained  sub-problems  with  many  fewer  variables,  and  has 
been  implemented  and  tested  extensively  on  the  IBM-7044.  Computational 
experience  is  presented  showing  that  large  problems  can  be  solved  in 
reasonable  time  with  good  accuracy,  especially  when  compared  with  the 
performance  of  a  general  nonlinear  progranning  algorithm  applied  directly 
to  the  equivalent  concave  program.  These  results  should  also  interest 
those  concerned  with  solving  large  structured  nonlinear  programs 
(the  largest  we  have  solved  involves  381  variables  and  123  linear 
constraints),  since  the  reduction  strategy  used  here  is  of  quite 
general  applicability. 
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I.  INTRODUCTION 

Let  the  random  variable  X'^  have  a  (right-continuous)  cumulative 

distribution  function  ,  j  =  1 . N,  independent  and  stochastically 

*  1  2  N 

ordered  according  to  F  ^  F  i  ...  i  F  .  Maximum  likelihood  esti- 

A  1 

mates  F  of  the  distributions  are  desired,  based  on  the  independent 
observations  x|  (j  -  1,...,N;  i  =  l,...,n^)  subject  to  the  constraint 
F  i  . . .  i  F  . 

a1  a2 

When  N  =  2,  explicit  formulae  for  F  and  F  were  derived  by 
Brunk,  et  al.  [2].  They  were  unable  to  extend  their  results  to  the 
case  N  ai  3.  This  study  makes  use  of  the  Kuhn-Tucker  Theorem  of  concave 
programnlng  [3]  to  furnish  an  alternate  derivation  of  the  results  In 
[2],  and  to  derive  a  partial  characterization  of  the  solution  for 
the  case  N  %  3.  We  then  show  how  the  author's  reduction  strategy 
[4,5]  can  be  used  to  take  advantage  of  the  analysis  to  achieve  an 
effective  computational  solution  for  N  ^  3  and  large  numbers  of 
observations.  The  computational  procedure  reduces  the  constrained 
problem  to  a  short  sequence  of  essentially  unconstrained  sub-problems 
with  many  fewer  variables. 

We  begin  by  reformulating  the  given  problem  as  one  of  concave 
programming  with  a  finite  number  of  variables  and  constraints.  The 
analysis  of  the  resulting  concave  program,  with  the  help  of  the  Kuhn- 
Tucker  Theorem,  is  carried  out  in  Sec.  III.  In  Sec.  IV  we  describe 

We  write  "F  i  G"  for  two  distributions  if  F(x)  2  G(x)  for  all  x. 
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a  computational  method  based  on  the  reduction  procedure.  The  method 
has  been  Implemented  and  tested,  and  in  Sec.  V  we  give  computational 
evidence  that  it  is  very  effective  in  terms  of  computing  time, 
storage  requirements,  and  numerical  accuracy.  Details  regarding 
the  program,  ES0D*3,  are  available  in  [l]. 
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II.  TRANSFORMATION  OF  THE  PROBLEM  INTO  A  CONCAVE  PROGRAM 


The  constrained  maximum  likelihood  estimate  F*^  must  be  sought 
from  the  class  of  discrete  distributions,  or  the  likelihood 
product  has  no  upper  bound.  Let  f(x)  be  the  probability  assigned  to 
the  value  x  by  a  discrete  distribution  F(x).  Then  we  must  maximize 
the  likelihood  product 


N  n-^ 

n  n  f^(xJ) 

j-i  1-1 


over  all  discrete  distribution  functions  with  the  desired  stochastic 
ordering.  The  task  of  this  section  is  to  reduce  this  problem  to  one 
with  a  finite  number  of  variables  and  constraints. 

For  j  *  1,...,N,  define  the  following:  x^  <  x;|  <  . . .  <  x^  to  be  the 

14  Til 

11  J 

distinct  values  taken  on  by  the  X^;  x^  to  be  the  smallest  of  the  for 

any  J  such  that  j  ^  J  ^  N;  and  x^  to  be  the  largest  of  the  for  any 

J  such  that  1  ^  J  ^  J.  We  assert  that  It  suffices  to  seek  the  maximum 

of  the  likelihood  product  over  all  discrete  distributions  having  the 

desired  stochastic  ordering  and  also  having  the  additional  property  . 

Property^ :  For  J  -  1 . N ,  f^  (x)  -  0  unless  x  €  fx^,  x|,...,x^ 

This  assertion  follows  from  lenina  1. 

Lenna  1:  If  F^ , . • •  ,F^  are  any  discrete  distributions  stochastl- 
'  ~  o  0 

1  N 

cally  ordered  as  F  a  ...  2  F  ,  then  there  exist  discrete  dlstrl- 
butions  f]^,...,F^  stochastically  ordered  as  F^  2  ...  2  with 
property  ^  and  at  least  as  large  a  value  of  the  above  likelihood 
product . 

Proof:  We  shall  generalize  an  argument  used  In  [2].  Define 
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f!(x)  - 


I 


,  if  X  <  X 

’  o 

■),  It  xj  £  X  <*U’  ‘  "  “ . 


,  if  X  2  X 


and 


,  if  X  <  x^ 

o 

F^(x)  -  Min  {FoCxJ+I-).  F;J“^(xJ)j  ,  If  x|  <  x  <  xj^^  ,  i  -  0, 


,  if  X  2  X 


nj4 1 


for  j  ■  2,...,N,  where  it  is  understood  that  any  vacuous  intervals  in 

1  2  2 

the  definition  of  the  Ft  are  to  be  ignored  (e.g.  ,  if  x  =  x  ,  then 

X  0  1 

2  2  i 

fx  ,  X,)  is  empty).  Clearly  the  have  property  ^and  the  desired 
o  I  * 

stochastic  ordering.  To  see  that  the  are  true  (increasing) 
distribution  functions  and  that  they  have  at  least  as  large  a  likeli¬ 
hood  product  as  the  F^  ,  it  suffices  to  show  fi(x;|)  2  f^(x'l)  for 
i  ■  1  , . . .  .Oj  and  j  *  This  is  clearly  true  for  j  ®  1. 

Consider  now  2  «  j  <  N.  Note  that  F^  2  F^  by  construction,  which 

implies  F*  2  F^ ,  which  implies  F^  2  F^ . which  implies  F*  2  F^. 

X  O  X  O  X  Q 

Thus  for  1*0,  1 . n^,  we  have 


j "I j ■ 


.J/J' 


since 


It  follows  that  for  i  -  l,...,n^. 


fi(xj)  *  Fi(xJ)  -  Fi(xJ-)  2  F^(xJ)  -  Fj(xj-)  =  f^(xj) 


rJ  (J. 


This  completes  the  proof 
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Thus  we  have  reduced  the  problem  Co  one  with  less  than  E  (n  +2) 

j-1  J 

variables:  f  (x^)  ,  f^(x| +1^^'^  ■  1 ,  • . .  iN)  .  We  say  less 

i  "^11  j 

than  because  some  of  the  x  coincide  with  x,  ,  or  x"^  . ,  with  x"^ 

o  1  n  +1  n, 

1  1  N  N  ^ 

(e.g.,  it  is  ob/ious  that  x  . i  =  x  and  x  =  x, ) .  For  notationai 

'  ®  ''l  °  ^ 

convenience  we  shall  adopt  Che  convention  that  the  redundant  end 

numbers  among  x^  <  x^  <  x;)  <  . . .  <  x^  x^  , ,  have  been  eliminated 
°  o  1  2  nj  n^+1 


and  that  n^  has  been  redefined  where  necessary  so  that  the  remaining 

x|  can  be  relabeled  x|  . . .  <  x^  .  The  corresponding  f^(x|)  are 

1  i 

denoted  by  yj| . (j  *  1,...,N). 


The  constraints  are  of  two  varieties.  The  first  requires 
the  y|  to  correspond  to  discrete  probability  distributions  j  all  y| 
must  be  nonnegative,  and 


(l.j) 


1 4 


must  hold  for  j  =  1,...,N.  The  second  variety  assures  stochastic 
ordering  ;* 


(2.j) 


14  *  [i  =  -i**  <  -m]  • 


k  *  1,, 


•  ”1  I 


for  j  -  1,...,N-1.  (For  the  sake  of  a  linear  notation,  complicated 
restrictions  of  the  domain  of  summation  follow  a  colon  immediately 
following  the  suranand.) 


*(2.j)  can  be  Interpreted  as  enforcing  F'^  >  F'^'*’^  by  requiring 


F^(xJ^l-)  2  k  -  1,...,  n^-l. 


A  similar  interpretation  can  be  offered  for  (3.j). 
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An  alternative  to  (2.j)  Is 


(3.J) 


y  yi  :  xj  <  =<^■'“1  *  I 

.1  J  i-1 


n  . "j+r^ 


For  the  sake  of  being  specific  we  shall  use  (2.j)  in  the  sequel. 

Letting  h|  be  the  number  of  X'^  observations  with  value 
Xj  (h^  or  h'^  nay  be  0)  and  using  log  likelihood,  we  have  finally 

11  ttj 

reduced  our  original  problem  to  the  concave  program: 

N  J 

V'  V  1  1 

(P)  Maximize  yi  subject  to  (l.j),  j  =  1,...,N 

yj  z  0  j*l  1=1  J  /o  •  \  -1  „  1 

■'1  and  (2.j)  ,  J  =  1  .  ,N-1. 


A  number  of  constraints  among  (2.j),  j  =  1,...,N-1,  are  likely 

* 

to  be  manifestly  redundant,  and  can  therefore  be  eliminated;  for 
each  j  ,  denote  the  k-indices  of  the  remaining  nonredundant  constraints 
by  .  Moreover,  certain  of  these  nonredundant  constraints  can 
actually  be  written  as  equalities  ,  as  we  now  show. 

Lenana  2;  For  j  -  1,...,N, 

A.  h|  ■  0  implies  1  €  and  the  corresponding  constraint  is 
satisfied  with  equality  in  any  optimal  solution  of  (P) . 

B.  h"^  =0  implies  that  the  right-hand  side  of  the  last  con- 

J  1.1  1  4 

straint  in'^''  is  (yj^  +  . . .  +  yj|  and  that  this  constraint 
is  satisfied  with  equality  in  any  optimal  solution  of  (P) • 
Proof;  By  construction,  h|  can  vanish  only  for  j  €  fl,...,N-l}, 
and  when  it  does,  x^  >  necessarily  holds.  This  implies  that  the 

first  constraint  of  (2.j)  is  y|  ^  y|^^  +  where  we  have  not 

* 

It  is  a  trivial  matter,  due  to  the  very  special  structure, 
to  identify  the  redundant  constraints. 
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specified  the  right-hand  side  fully.  This  constraint  is  clearly  not 
redundant,  so  1  €71^.  Suppose  now  that  this  constraint  is  amply  satis¬ 
fied  at  an  optimal  solution  of  (P).  Then  y|  can  be  decreased  by  a  posi¬ 
tive  amount  and  y^  increased  by  the  same  amount  without  destroying  feasi¬ 
bility  in  (P) ,  as  one  can  easily  verify.  But  this  improves  the  value 
of  the  loglikelihood  function,  which  is  impossible.  This  proves  part  A. 
By  construction,  li'^  can  vanisli  only  for  J  C  f2,...,NT,  and  when 

J  J-1 

it  does,  X  1  <  ^  necessarily  holds.  This  implies  that  the  last 
”l-l 

J  J  *■  1  1  X  1 

nonredundant  constraint  of  Is  y^  +.  . .  ^  y^  +...+  y^ 

n^-t 

where  we  have  not  specified  the  left-hand  side  fully.  Suppose  now 

that  this  constraint  is  amply  satisfied  at  an  optimal  solution  of  (P). 

Then  ,  can  be  increased  by  a  positive  amount  and  decreased  by 
n.-l  "^n,  ■' 

J  J 

the  same  amount  without  destroying  feasibility  in  (P)  ,  as  one  can 

easily  verify.  But  this  improves  the  value  of  the  loglikelihood 

function,  which  is  impossible.  This  proves  part  B. 

For  j  =  1,...,  N-1  ,  let  denote  the  constraints  of  identified 

in  Lemma  2.  The  results  of  this  section  are  summarized  and  extended  in 

Theorem  1. 


Theorem  1 ;  The  original  constrained  maximum  likelihood  problem 
can  be  formulated  as  the  concave  program  (P) ,  which  has  a  unique 
optimal  solution  (y^i.  Each  component  of  this  solution  is 

positive,  and  it  satisfies  the  constraints  in  (j  -  1 . N-1) 

with  strict  equality. 

Proof ;  The  only  assertion  unproved  thus  far  is  that  (P)  has  a  unique 
optimal  solution  and  that  this  solution  has  all  positive  components. 

It  is  easy  to  construct  a  feasible  solution  of  (P)  with  each 
component  positive.  For  example,  it  is  readily  verified  that  the 
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followlng  construction  Is  possible:  let  =  1/n^  (i  =  l,...,np; 

2 

let  yj^  (i  ■  l.-.-.n^)  have  any  positive  values  summing  to  1  con¬ 
sistent  with  the  satisfaction  of  the  constraints  in  considered  as 

N 

equalities;...;  let  y^  (i  =  l,...,n^)  have  any  positive  values 

-i/iN-1 

summing  to  1  consistent  with  satisfaction  of  the  constraints  in  ^ 
considered  as  equalities.  Let  v  be  the  value  of  tlie  objective  function 
of  (P)  for  some  positive  feasible  solution.  Clearly  v  >  -  co.  Thus 

N  4 

J  j  i 

we  may  impose  the  additional  constraint  v  V  h .  In  y;:  s  v(>  -  oo) 

j=l  1=1 

on  (P) .  Since  (l.j)  places  an  upper  bound  of  1  on  each  y^  and 
thereby  an  upper  bound  of  0  on  In  y^,  this  constraint  requires  that 
h|  In  y|  5  v(>  -  ®)  for  all  I  and  j.  For  h^  >  0  ,  this  requirement 
is  equivalent  to  y|  >  0).  From  Lemma  2  and  constraints 

(l.j)  it  follows  that  y|  is  bounded  strictly  away  from  0  even  when 
h|  *  0  (in  this  case  the  above  requirement  is  vacuous,  since  v  <  0) . 

All  this  is  to  say  that  the  y|  may  be  taken  to  be  bounded  strictly 
away  from  0  in  (P) .  By  the  continuity  of  the  maxlmand  on  the  compact 
thus-restricted  feasible  region,  (P)  is  seen  to  have  a  positive 
optimal  solution  fy|} .  From  the  strict  concavity  of  In  y|  we 
conclude  that  y^  is  unique  when  h^  >  0.  This  implies,  with  the  help  of 
Lemma  2,  that  y|  such  that  1\|  *  0  is  also  unique.  This  concludes  the 
proof . 

We  remark  that  (P)  is  amenable  to  solution  by  various  existing 
concave  programming  algorithms.  For  example,  the  algorithms  of  Graves 
[6]  and  Rosen  [7]  seem  appropriate.  So  does  a  family  of  approximate 
algorithms  of  the  "separable  programming"  variety,  in  view  of  the 
linear  separability  of  the  objective  function.  (See,  e.g.,  Miller  [8].) 
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However,  these  algorithms  do  not  take  full  advantage  of  the  important 
special  structure  enjoyed  by  (P) . 
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III.  APPLICATION  OF  THE  KUHN-TUCKER  THEOREM 

In  this  section  we  apply  the  Kuhn-Tucker  Theorem  [3]  to  (P)  to 
recover  the  analytical  solution  of  Brunk,  et  al .  [2]  for  the  case  N  >  2. 
We  shall  see  that  the  method  becomes  unwieldy  for  the  case  N  2  3,  but 
nevertheless  does  yield  analytical  results  leading  to  an  effective 
computational  procedure. 

The  Kuhn-Tucker  Theorem  can  be  applied  to  (P) ,  in  view  of  the 
results  of  the  last  section,  to  yield  the  following  characterization 

o£  (y^). 

Theorem  2;  A  trial  solution  {y^l  is  an  optimal  solution  of  (P) 
if  and  only  if  all  y^  >  0,  and  for  some  collection  S^(j  “  1,...,N-1) 
satisfying  C  C  there  exist  generalized  Lagrange  multipliers 

X^(k  f  S'*  ;  j  »  1 . N-1)  and  u'*(j  =  1 . N)  such  that 

(S"^;  y^ ;  satisfies  the  corresponding  Kuhn-Tucker  con¬ 

ditions: 

/  N  "j  N  "j 

(KI-l)  I  Th|  in  yj  +  ^,.1  (^y|  -  l) 

Vj-l  1-1  J-1  1-1 


J=l  k€s^ 


=  0  for  all  i  and  i, 


i=l 
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(  "  J 

2  yj  =  1 .  j  -  1. •  -  ..N 

i=l 


(KT-2)  \ 


\  J  -  V  J+1.  J+i  .  J  u  c  oi  . 

L  /.  •  i  k+1’  ^  •  J 

1-1  t 


l....,N-l, 


(KT-3) 


I  ^1  ^  I  ^  4+1’  . . N-1, 


1=1  1 


(KT-4)  >j  2  0,  k  €  j  -  1,...,N-1. 


Our  general  strategy  will  be  to  determine  from  (KT-1)  and  (KT-2)  , 
for  S'^  satisfying  5'^  £  what  {y^}  and  {\j|]  must  be.  For 

(KT-3)  and  (KT-4)  to  be  satisfied,  it  will  follow  that  the  S"^  must 
satisfy  certain  necessary  and  sufficient  conditions.  With  the  so 
determined,  it  remains  only  to  compute  {yj|]  from  (KT-1)  and  (KT-2). 

This  strategy  will  be  completely  successful  only  for  the  case 
N  =  2.  We  assume  henceforth  that  c  (J  .  1 . N-l). 

3.1  THE  CASE  N  =  2 

We  shall  simplify  the  notation  somewhat  in  this  subsection  by 
writing  S  for  s\  for  X^,  and for  7? 

When  the  differentiation  is  performed,  (KT-1)  becomes 

(KT-l)J  (h[/y[)  +  ^  X,^;k  2  i  =  0,  i  =  l,...,n^, 

^  k6S  * 
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(KT-l)J  (hJ/y^)+^i^-  i  =  l....,n 


2* 


k€S 


Let  S  be  expressed  as  , . . .  ,k^}  ,  where  kj^  <  ...  ,  when  S 

is  not  empty.  We  see  from  (KT-1)  that  when  S  Is  empty,  the  ratios 

(h|/y^)  are  constant  for  1  ■  and  j  “  1  «nd  2.  When  S  Is 

not  empty,  (hj/yJ)  remains  constant  over  the  Integer  Intervals 

2  2 

n  ,  kj^  ]  (kj^ ,  k2],...,(k^,  while  (h^/y^)  remains  constant  over 

2  1  1  2  1 

the  integer  intervals  [j  :Xj  <  ,  {j  :x^  «-  x^  <  \  . 

1  21  1^2^  ^ 
fj:x,  <  X  <  +1^’  ■’Sc  +1  ^  ^  Induces  a  partition 

^s-1  ^  s  *^s  ^ 

P  of  fl,...,nj^l  and  a  partition  Q  of  fl,...,n2}.  We  denote  a  generic 

cell  of  P  by  p  ,  and  the  i*"^  cell  by  P^Ci  '  l-»'-".s+l);  similarly,  we 
use  the  notations  q  and  for  Q.  Since  there  is  a  natural  1:1  corre¬ 
spondence  between  the  cells  of  P  and  Q,  we  sometimes  denote  the  cell 
of  Q  corresponding  to  cell  p  of  P  by  q(p). 

Thus  to  determine  ry|]  satisfying  (KT-1),  it  suffices  to  determine 

the  2(s+l)  ratios  (h|/y|)  corresponding  to  the  cells  of  P  and  Q. 

1  2 

Using  an  obvious  notation,  we  denote  these  ratios  by  (h/y)^  and  (h/y)^. 


Leana 


_3:  If  (KT-1)  has  a  solution  Ty^l,  then  for  each  p  €  P, 


(h|/y^)  “  (h/y)p  for  all  i  €  p,  and  for  each  q  P  Q,  (h^/y^) 
(b/y)q  for  all  1  P  q. 


Let  us  turn  now  to  (KT"2)  .  With  our  new  notation,  we  may  rewrite 
(KT-2)  as  (KT-2A)  and  (KT-2B)p  for  each  p  €  P: 


(KT-2A) 


I  ^i  “ 


(KT-2B) 


In'  In 

lep  i€q(p) 


By  Lemma  3,  (KT-2B)  can  be  written 

P 


(KT-2B) 


(y/h)J  )n‘  =  (y/h)^ 


I 


i-€q(p) 


where  it  is  understood  that  if  =  0,  then 

(y/h)J  Jk;, 

which  is  undefined,  is  defined  as  simply  y|;  and,  similarly,  that  if 

h^  =  0,  then 
"2 


(y/h)' 


s+l 


I 


is  defined  as  simply  y 

"2 

Thus  far  we  have  reduced  (KT-1)  and  (KT-2)  to  [2(8+1)  +  (s+l)  +  l] 

12  1 

independent  equations  (n5unely,  (KT-1).  and  (KT-1),  for  k  ^  S,  (KT-1)  , 

2  ^ 
(KT-1)  ,  (KT-2b)  for  p  €  P  and  (KT-2A))  in  [3  (s+l)  +  s  +  2]  unknowns 

**2  P 

1  2 

(namely,  y^  (respectively  y^)  for  one  1  in  each  cell  of  P  (respectively 
k  12 

Q)  ,  X  for  k  ^  S,  and  |j,  ,  p.  ) .  We  shall  see  that  we  can  obtain  a 


complete  solution. 
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It  remains  to  consider  the  case  S  From  (KT-1)^  we  have 

"j 

for  j  =  1 ,  2 . 

(5)1  ^1  =  .(hi  /yl  ). 

J  J 

Upon  subtracting  (KT-1)^  from  (KT-1)^  ,  one  obtains 

\ '  K  +i'i'k.+i)  -  >•  1  ■  1 . . 

ill  i  i 

1  2 

From  (KT-2B)  ,  (5)  ,  and  (6)  we  see  that  we  can  easily  express  ij,  ,  u  , 

2  2  1 

X.  (k€S)  ,  (y/h)  (i  =  l,...,s)  and  y  in  terms  of  y  (k€S)  and 
It  02  ic 

y  .  The  problem  of  obtaining  a  complete  analytic  solution  to  (KT-1) 
"l 

and  (KT-2)  thus  reduces  to  solving  for  the  last  mentioned  quantities. 

I  2 

For  each  p  ^  P,  add  (KT-l)^^  and  (KT-l)^  for  some  i  p  and 
J  €  q(p)'  The  result  is 

(7)  (h/y)J  +  /  +  =  0.  P  ^  p. 

Substituting  (KT-2B)  and  (5)  into  (7)  ,  one  obtains  for  each  p  p 


(y/K)J 


+  (h 


/y') 
"2  "2 


where  it  is  understood  that  if  hj^  “  0,  then  (8)^  is  replaced  by 

yj  -  )  hj/fch^  /y^  )  +  (h^  /y^  )  . 


i€q. 


Equation  (KT-2B)  can  be  used  to  eliminate  from  (8).  The 
s+1  .  2 

resulting  expressions  for  y.  (k  €  S),  in  terms  of  y  ,  can  be  used  with 

K  n « 

(Kr-2A)  to  solve  for  y  .  The  result  is 


where 


Tlj  -  ^  h|  (j  *  1 . s+1), 


<  -  Y  (j 

j  ^  ^ 

i€q, 


-  1 . 8+1),  and  V  =  ^  Tl|  (j  *  1,2) 


Finally,  we  liave  from  (8)  and  (9)  for  i  ■  1 . s+1 


(y/h) 


1  <<  ^ 

^i  (Tl^  +  Tl^)Tl! 


where  it  is  understood  that  if  h^  ■  0,  then  (10) is  replaced  by 
12  12 

yj^  ■  T]  ) .  This  is  the  desired  analytic  solution.  For 

completeness,  we  record  (using  (10)  in  (KT-2B))  that  for  i  -  1,...,8+1 


(y/h)q  -  (TlJ  +  T]J)/(V  +  Tl^)Tli 


where  it  is  understood  that  if  h^  ■  0,  then  replaced  by 

>''2  • ’ilW  ^ ’i')- 
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Let  us  sununarlze  our  results  as 

Lemma  4;  Let  S  satisfy  "5  C  S  c77.  Then  (KT-1)  and  (KT-2)  have 
a  unique  solution  (y|;  given  by  (10)  and  (11),  (6)  and 

(5).  ((4)  is  subsumed  by  (10),  (11)  and  ( ))  in  the  event  that 

S  “  ^.)  Moreover,  all  >  0  in  this  solution. 

Now  that  we  have  an  analytical  solution  of  (KT-1)  and  (KT-2)  for 
any  S  satisfying  ^  S  C"^,  let  us  see  what  furtlier  conditions  must 
be  imposed  on  S  so  that  this  solution  also  satisfies  (KT-3) 
and  (KT-4). 

Consider  (KT-4),  wltich  can  be  written 

(Kr-4)  aO,  i“l,...,s. 

i 

In  view  of  (6)  and  (10),  (KT-4)  is  equivalent  to 

(V  +  (V  + 

1  2  *  1  2  i“3,...,s, 

*  \+i>  (’ll  +  \+i> 

or 

(12)  (7lJ/Tl[)  i  (Tli+i/Tl[+i).  i  “  l,...,s. 

Consider  now  (KT-3),  which  can  be  written 

k 

(KT-3A)  ^  yj  k  ^  S. 

i-1 


1 


-18- 


For  the  sake  of  simplicity,  we  have  not  bothered  to  exclude  the 
redundant  constraints  among  (2.1).  In  view  of  (KT-2),  (KT-3A)  amounts 
to  requir.-.-s  •'or  i  *  l,...,s+l  and  all  k  ^  S  but  k  f  , 


^  m 

k 

• 

r*  1 

V  2  2^1 

I 

2 

1  yj  <  \+i 

Using  (10),  (11)  and  Lemma  3,  tliis  becomes 


(Tl|  +  Ij.) 

Tl|(V  +  Tl^) 


^  k 


!■  1  2 

cn'  +  Tip 

2  1  2 

\(T1  +  T) 


I 


h2 

J 


2 
X  . 

J 


<  X 


1 

k+1 


for  i  =  1 , .  . .  ,,s+l  and  k  f  p^^ .  For  i  =  1 . s+1,  a  little 

manipulation  yields 


(13) 


I 


,22^1 

J  J  k+1 


I 


,  1  ■ 

li . :  j  ^  k 
J 


for  all  k  e  p  but  k  ^  S 
t 


We  have  proven,  in  view  of  Lemmas  3  and  4 
Lemma  5;  Let  S  satisfy  ^  c  S  g.7l  . 

A  necessary  and  sufficient  condition  on  S  in  order  that  the 
unique  solution  of  (KT-1)  and  (KT-2)  also  satisfy  (KT-3)  and 
(KT-4)  is  given  by  (12)  and  (13). 

Lemmas  3,  4,  and  5  immediately  yield,  in  view  of  Theorem  2,  the 
central  result  of  this  section. 

Theorem  3;  Let  S  satisfy  9  S  c7(  and  also  (12)  and  (13). 
Then  the  optimal  solution  {y^}  of  (P)  is  given  by  (10)  and  (11). 
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Brunk,  et  al.  [2]  has  given  a  simple  geometrical  method  for  con¬ 
structing  S  so  as  to  fulfill  the  conditions  of  Theorem  2.  Conditions 
(10)  and  (11)  have  purposely  been  written  in  sucli  a  form  as  to  make 
this  construction  clear.  It  will  not  be  repeated  here,  since  it  cannot 
be  generalized  to  the  case  of  primary  interest  to  us. 


3.2  THE  CASE  N  ^  3 

Whan  the  partial  differentiation  in  (KT-1)  is  carried  out,  the 
result  is 

(KT-1)[  (h[/y[)+u^+  V  X^:k  >  i  =0,  t  = 

kfS^ 


(KT-1)^ 


(h|/yj)  + 


7  + 

rcJ  ^ 


i  =  1  , .  .  .  .n^  ;  j  *  2  , .  . .  ,N-1, 


/„.r  .xN  jv  ^  N  V  r,N-l  N  ^  N-n 

(KT-l).  ,!>k  =’‘l'Vi  "■  . "n' 


Let  be  expressed  as  [kj,..,,k^  },  where  k|  <  .  . .  <  k^  ,  when 
is  not  empty.  We  see  from  (KT-1)  that  the  ratios  (li^/y^)  remain 
constant  over  certain  integer  intervals.  Toward  identifying  tliese 
intervals,  we  make  tlie  following  definitions.  Let  P’^  be  a  partition 
of  fl,...,nj}  induced  by  S  for  J  “  1,...,N-1,  as  follows:  [l,  k^], 

(k|,  k^],...,(k^  .  Let  Q-^  be  a  partition  of  fl . "j+1^ 

induced  by  for  j  =  1,...,N-1,  as  follows  (for  better  legibility, 
complicated  subscripts  are  placed  in  parentheses) : 


{iixj^'*’^  <  x-^(k^+l)  ],  fi  :x^(k|+l)  <-  <  x^(k^+l)}  ,. .  .  , 


fl-:x^(kg  j^+l)  ^  xj'^  <x^(kg  +1)]  ,  fi:x-’(k^  +1)  ^x|''’^]. 

J  j  j 
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For  j  ■2,...,N-1,  let  ^  be  the  product  partition  of 

determined  by  P'^  and  Let  and  We  denote  a  generic 

cell  of  P'^ ,  Q'^,  or  0'^  by  a  subscript,  and  define  Tv|=  T  ,h;|  for  each 
p  €  n  and  j  *  1, . . . ,N. 

From  (KT-1)  we  see  that  (li|/y^)  remains  constant  within  each  cell 
of  ,  j  ■  1,...,N.  Therefore,  to  determine  [y|}  satisfying  (KT-1) 
it  suffices  to  determine  the  ratios  of  (li^/y^)  within  these  cells. 

We  denote  tliese  ratios  by  (li/y)p.  The  symbol  (y/h)^,  used  in  tlie 
sequel,  denotes  tlie  reciprocal.  We  have 

Lemma  3A:  If  (KT-1)  has  a  solution  {yi},  then  (h|/y|)  =  (h/y)p 
for  each  i  in  cell  p  of  j  “  1,...,N. 

We  turn  now  to  (KT-2).  It  can  be  written  as  (14)  together  with 
( 15)p  for  each  p  ?  P-^,  j  =  1,...,N-1: 


In 


I  >'i  ■  I  '■f 


By  Lemma  3A,  (14)  may  be  rewritten  as 


(KT-2A) 


and  (15)^  as 
P 


(KT-2B) 


i^‘(y/h)J  -  I 


n«P^Q‘ 
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(KT-2B)- 


I  Ti^(y/h)^  ■  I 


rtpJ.QJ'* 


P 


for  j  =  2,...,N-2  (for  N  2  4) ,  and 


(KT-2B) 


V  /.  ,1 


n€p"-‘.Q“-' 


This  assumes,  of  course,  that  all  hj  >  0,  since  <y^/!i^)  is  undefined 
otherwise.  When  h|  ■  0  (this  can  happen  only  for  i  *  I  or  i  -  n.)  the 
corresponding  term  in  (14)  or  (15)  should  replace  the  offending  term 
in  (KT-2A)  or  (KT-2B)^;  that  is,  replaces*  the  corresponding  Tljj(y/h)j. 
Let  us  see  what  can  be  done  to  solve  (KT-1)  and  (KT-2)  for  the 
Xk.  and  y^.  From  (KT-1)^  we  see  that 

*^1  nj 

(16)  -  -(h/y)^  ,  j  =  1 . N. 

j 

Upon  subtracting  (KT-1)^^  from  (KT-l)^j,  one  obtains  for  i  =  1 . s 

k  +1  k  1 


x\  -  /y\  )  - 

k,  k.+l  k.+l  k  k 

ill  11 


One  similarly  obtains  for  j  =  2,...,N-1  and  i  =  l,...,Sj, 


It  is  helpful  to  recall  that  our  assumption 
(j  -  1 . N-1)  implies  that  when  h|  =  0,  we  have  =  0  for  the  cell 


containing  i. 
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Iy\  )  - 

k^+l  k-^+1  k;^  k^ 

111  i  1 


^  I  'r'-v-k;; ■ 

wsJ->  '■i  "!« 

Thus  one  can  express  the  and  in  terms  of  the  y^. 

By  an  appropriate  linear  transformation  of  the  system  (KT-1), 
one  can  eliminate  tlie  and  Xj|*  Tlie  resulting  system,  taken 
with  (KT-2A)  and  (KT*2B),  is  nonlinear  and  does  not  yield  to  analytical 
solution  techniques.  This  Is  In  contrast  to  the  case  N  =  2,  and 
destroys  our  hopes  of  obtaining  generalizations  of  Lemmas  and 
3  for  N  2  3,  Lemna  3A  and  (17)  will  be  of  considerable  value  In 
what  follows,  however,  as  will  the  following  weaker  version  of 
Lenina  4. 

Lemma  4A;  Let  satisfy  E  E  V  »  J  *  1,...,N-1.  Then 
(Kr-l)  and  (lcr-2)  have  a  unique  solution  (yj;  X^;  )  .  Moreover, 

all  y|  >  0  In  this  solution,  and  the  and  X^  are  given 
in  terms  of  the  y^  by  relations  (16)  and  (17),  respectively. 

Proof;  Consider  the  following  subproblem  derived  from  (P)  according 


to  (j=l . N-1)  ,  where  2^  c  (Z~y^  ; 


N  n, 


Maximize  ^  ^  y|  subject  to  (KT-2)  . 


yj  2  0  j=l  i=l 
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The  proof  of  Theorem  I  is  ea8ll>  adapted  to  show  that  (P  )  has  a 

d 

unique  optimal  solution,  and  that  this  solution  has  all  positive 
components.  Applying  the  Kuhn-Tucker  Theorem  to  (P-)  as  in  Theorem  2, 
we  see  that  (KT-l)  and  (KT-2)  are  in  fact  the  Kuhn-Tucker  conditions 
associated  with  (P„) .  Hence  (KT-l)  and  (KT-2)  must  have  a  solution, 
and  fy^}  is  unique  and  positive  in  any  such  solution.  The  above 
manipulations  show  that  the  and  must  also  be  unique,  since 

they  can  be  expressed  in  terms  of  the  y|  according  to  (16)  and  (17). 
This  completes  the  proof . 
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IV.  A  REDUCTION  PROCEDURE 

The  basic  strategy  of  the  last  section  was  to  determine,  with  the 
help  of  a  general  solution  of  (KT-l)  and  (KT-2) ,  conditions  that 
the  S'^  must  satisfy  (in  addition  toH'^  c  S'^  ^  7t^)  so  that  (KT-3) 
and  (KT-4)  will  have  a  solution.  With  the  S'^  so  determined,  the 
optimal  solution  of  (P)  is  obtained  from  the  corresponding  solution 
of  (KT-l)  and  (KT-2).  This  strategy  was  completely  successful  only 
in  the  case  N  ■  2  ,  the  main  obstacle  for  the  case  N  2  3  being  the 
apparent  impossibility  of  obtaining  a  general  solution  of  (KT-l)  and 
(KT-2).  In  this  section  we  essentially  pursue  the  same  strategy,  but 
in  an  iterative  numerical  manner.  Lemmas  3A  and  4A  will  be  quite 
valuable  in  this  pursuit,  and  the  result  is  an  efficient  computational 
procedure . 

The  proof  of  Lemma  4A  indicates  that  solving  the  concave  program 

(Pg)  Maximize  ^  h|  In  y|  subject  to  (KT-2) 

yf  >  0  j=l  i=l 

is  equivalent  to  solving  (KT-l, 2).  Our  strategy  then  is  to  choose 
an  initial  S  and  solve  (P^) ,  and  determine  whether  the  resulting 
(y|;  i  satisfies  (KT-3)  and  (KT-4);  if  so,  then  the  current 
solution  solves  (P)  and  we  may  terminate  ,  but  if  not,  then  S  is  modi¬ 
fied  according  to  the  sign  reversals  found  in  (KT-3)  and  (KT-4),  and 
the  new  (P_)  is  solved,  and  so  on.  After  a  usually  finite  number  of 
iterations ,  a  subproblem  yielding  the  optimal  solution  of  (P)  is 
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fouad.  A  generalization  of  this  strategy  has  been  presented  by  the 
author  elsewhere  [4,  5].  When  the  results  therein  are  specialized 
in  view  of  Leima  4A  to  the  present  problem,  the  following  reduction 
procedure  and  theorem  are  obtained. 

Reduction  Procedure 

Step  1  :  Choose  an  initial  S  to  satisfy  ^ 
j  =  1 . N-1. 

Step  2  ;  Solve  (P^)  for  its  unique  optimal  solution  {y^}- 
Compute  by  (17).  Check  (lCT-3)  and  (lCT-4)  : 

if  both  are  satisfied ,  then  terminate 
^(y|]  solves  (P)y;  otherwise,  let  ( j  ■  1,...,N-1) 

be  the  indices  of  the  sign  reversals  found  in 
(KT-3,4),  that  is,  let  T-^  be 

[k  ^V-S^:  E  y|  <  I  }  U  [k  6  ;X^  <  O), 

and  go  to  Step  3 . 

j  *  i  J 

Step  3  ;  Pick  t  at  random  from  X  T'^ .  Replace  S'^  by  S'^  ±  t  , 

j 

where  j  is  the  index  satisfying  t  €  T  .  Return  to 
Step  1. 

Theorem  4;  The  reduction  procedure  is  well  defined,  and  ter¬ 
minates  in  a  finite  number  of  steps  with  probability  1. 

The  computational  efficiency  of  the  procedure  depends  on  (i)  the 
time  required  to  solve  a  typical  subproblem,  and  (ii)  the  number  of 
subproblems  that  must  be  solved  before  termination.  We  consider 
each  of  these  factors  in  turn. 

*S^  ±  t  means  S'^  -  t  if  t  €  S'^,  and  U  t  if  t  ^  S'^. 
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SOLVING  A  SUBPROBLEM 

The  subprobleras  have  the  same  objective  function  as  (P)  ,  but 
fewer  constraints,  and  these  are  equalities  rather  than  inequalities. 
Moreover,  the  result  of  Lemma  3A  makes  it  a  trivial  matter  to  sub¬ 
stantially  reduce  the  number  of  variables  (especially  when  S  is  sparse); 
all  that  is  needed  is  one  variable  for  each  cell  of  the  partitions 

. N)  Induced  by  S.  When  this  is  done,  the  constraints  (KT-2) 

can  be  rewritten  as  (KT-2A)  and  (iCT-2B)  ,  and  the  objective  function 
(after  a  simple  manipulation)  as 
N 

(18)  Y  In  (y/h)'^  +  a  constant, 

-  ^  ,  P  P 

J=1  p^qJ 

where  it  is  understood  that  if  h|  =  0,  then  the  term  Tj^  In  (y/h)p 

is  omitted  for  the  cell  containing  i  (cf.  the  next  to  last  footnote). 

P 

Hence  (P„)  ,  thus  simplified,  is  considerably  more  convenient  to  solve 
than  (P)  itself. 

Even  further  simplifications  are  possible,  in  fact,  by  solving 

the  constraints  (KT-2A)  and  (KT-2B)  explicitly  for  (Sj+l)  of  the 

(y/h)p^^  (J-1 . N-1)  and  one  of  the  (y/h)p.  Because  of  the 

convenient  triangular  structure  of  the  constraints  this  can  be  done 

easily  without  performing  a  matrix  inversion.  If  the  expressions 

for  the  dependent  variables  are  substituted  into  the  objective 

function,  only  the  nonnegativity  constraints  are  left  in  (Pg)  •  And  even 

these  can  be  ignored  if  (P  )  is  to  be  solved  by  an  optimal  gradient- type 

o 
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procedure,  for  In  (y/h)p  -  -  OD  as  (y/h)p  -  0  implies  that  boundary- 
repulsion  occurs  automatically.* 

All  this  is  to  say  that  (P_)  can  be  rewritten  as  an  essentially 
unconstrained  problem  in  many  fewer  variables  than  (P) .  Thus  the 
time  required  to  solve  a  typical  subproblem  can  be  expected  to  be 
very  small  compared  with  that  necessary  to  solve  (P)  itself  by  a 
comparable  gradient  method. 

THE  NUMBER  OF  SUBPROBLEMS 

It  seems  difficult  to  derive  theoretical  results  concerning  the 
number  of  subproblems  that  must  be  solved  before  termination.  The 
Markov  chain  analysis  and  computational  results  reported  by  this 
writer  in  [5],  however,  suggest  the  following  tentative  hypothesis: 
the  number  of  required  subproblems  can  be  expected  to  be  about  twice 
the  number  of  mistakes  made  in  identifying,  through  the  choice  of  the 
initial  subproblem,  the  constraints  of  (P)  that  are  actually  binding 
at  the  optimal  solution.  This  hypothesis  holds  provided  that  the 
sign  reversals  of  (KT-3)  and  (KT-4)  are  a  sufficiently  reliable 
guide  regarding  which  constraints  are  actually  binding  at  the  optimum. 

If  the  hypothesis  is  even  approximately  valid,  then  use  of  the 
reduction  procedure  is  greatly  encouraged;  for  it  has  been  our  experience 
with  numerous  test  problems  (cf.  the  next  section)  that  usually  only  a 

* 

We  assume  that  the  gradient  procedure  is  initiated  at  a  positive 
feasible  solution  of  (P„) ,  which  can  always  be  easily  done  (cf.  the 
proof  of  Theorem  1).  Strictly  speaking,  boundary-repulsion  only  occurs 
directly  for  the  ratios  actually  in  (18);  it  occurs  indirectly  for 

the  y|  such  that  h^  ■  0,  however,  because c  . 
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coap«r«tlvaly  mail  proportion  of  all  conitralntB  are  actually  binding 
at  the  optimal  solution,  and  hence  by  taking  as  the  Initial  S^,  one 
would  be  assured  that  the  number  of  subproblems  to  be  solved  Is  small. 
We  turn  now  to  the  computational  studies  of  these  matters. 
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V.  COMPUTATIONAL  EXPERIENCE 


IMPLEMENTATION 

Preliminary  tests  designed  to  study  the  tentative  hypothesis 
mentioned  In  the  previous  section  quickly  confirmed  that  the  sign 
reversals  found  in  (KT-3)  and  (KT-4)  at  Step  2  are  indeed  a 
remarkably  reliable  guide  in  identifying  the  constraints  actually 
binding  at  the  optinum.  To  take  fuller  advantage  of  this  situation, 
it  was  decided  to  implement  a  slight  variant  of  the  reduction  procedure, 
namely:  at  the  first  few  (five)  executions  of  Step  3  utilize  all  sign 
reversals  recorded  in  T^  to  modify  each  S"^,  instead  of  using  just  one 
at  random.  Theorem  4  obviously  remains  valid  for  this  variant. 

In  considering  how  to  solve  the  subproblems  (Pg) >  we  decided  to 
take  a  major  shortcut:  the  (triangular)  constraints  of  (P-)  were 
not  used  to  solve  for  and  eliminate  some  of  the  variables.  Undoubtedly, 
the  price  of  this  expediency  is  significantly  longer  solution 
times  for  the  subproblems,  but  it  was  adopted  for  the  sake  of  programning 
simplicity  since  the  primary  object  of  the  investigation  was  the  number 
of  subproblems  required  by  the  reduction  procedure. 

We  chose  a  general  nonlinear  programming  routine  due  to  Graves 
[b]  for  use  in  the  subroutine  for  solving  (P„) .  This  is  a  sophisticated 
large-step  gradient  method  that  has  proven  quite  effective  in  a  variety 
of  applications.  An  Important  issue  regarding  its  use  concerns  the 
choice  of  an  appropriate  termination  criterion.  Fortunately,  the 
routine  computes  an  upper  bound  v  on  the  true  optimal  value  of  (P_) 

D 

at  each  Iteration,  and  so  a  bound  on  the  maximum  possible  error  is 
always  available:  v  mirus  the  value  of  the  current  feasible  solution. 
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We  therefore  provided  a  parameter  XL  ("termination  level")  to  cause 
return  from  Graves'  routine  when  the  maximum  possible  relative  error 
for  (P  )  drops  below  XL  X  100  percent.  For  large  problems,  of  course, 
It  may  not  be  possible  to  reach  the  desired  value  of  XL  In  a  reasonable 
time.  Xhus  a  parameter  ICYCLE  Is  also  used  to  cause  a  return  from 
Graves'  routine  after  ICYCLE  Iterations. 

S  P.  Azen  carried  out  the  Implementation  In  Fortran  IV  for  the 
IBM  7044.  Details  of  the  test  program,  called  ESOD-3,  are  available 
In  [l]. 

XESX  PROBLEMS 

ESOD-3  has  been  tested  extensively  on  dozens  of  different  problems. 
All  have  been  solved  successfully.  We  present  in  Xable  1  only  our 
experience  with  some  of  the  larger  problems,  the  largest  of  these 
having  381  variables  and  123  nonredundant  constraints  (not  counting 
sign  restrictions  on  the  variables). 

Series  A  consists  of  three  distributions  (N  ■■  3)  ,  all  exponential, 
with  means  1,  2  and  3,  respectively.  Problem  Al  has  15,  12  and  9 
observations,  respectively,  from  each  distribution.  When  the  conversion 
Is  made  from  the  constrained  maximum  llklihood  estimation  problem  to  a 
concave  program  of  the  form  (P) ,  Al  has  37  variables  and  13  non- 
redundant  constraints.  Xhe  number  of  observations  from  the  three 
distributions  Increases  In  multiples  of  3,  4,  and  3,  respectively. 

In  problems  A2  through  A6. 
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Serles  B  consists  of  4  Weibull  distributions  with  scale  parameter 
X  and  shape  parameter  O'  as  follows: 

Distribution  X  a  Mean 

1  5  1  0.2 

2  1  2  0.88263 

2  1  3  0.89338 

^  0.1  1  10.0 

Problems  Bl  through  B4  each  have  10,  20,  30,  and  38  observations  from 
each  distribution,  respectively. 

Series  C  consists  of  10  normal  distributions  with  unit  variance 
and  means  0,  2,  4,  6,  7,  8,  9,  9^,  10,  10^,  respectively.  Problems  Cl 
through  C4  also  have  10,  20,  30,  and  38  observations  from  each 
distribution. 

Table  1  sunmarizes  our  computational  experience  with  these 
problems.  All  were  run  with  TL  ^  0.02,  and  with  the  initial 

Series  A  was  run  with  ICYCLE  ■  15,  series  B  with  ICYCLE  =  25, 
and  series  C  with  ICYCLE  ~  30.  These  values  of  ICYCLE  are  quite 
small,  and  cause  the  subproblems  to  be  considerably  suboptimized; 
this  was  done  deliberately  to  take  advantage  of  the  empirical  observa¬ 
tion  that  the  ultimate  sign  reversals  in  (KT-3)  and  (KT-4)  manifest 
themselves  long  before  (P^)  is  solved  optimally.  The  parameter  ICYCLE 
is  automatically  made  inoperative  in  the  final  subproblem,  so  that  TL 
becomes  effective  as  the  termination  criterion.  In  some  of  the 
larger  problems,  it  will  be  noted  that  the  2-percent  level  was 
not  achieved;  in  these  cases  the  permissible  step  length  SK  in 
Graves'  routine  became  0,  so  that  no  farther  improvement  was  numerically 
possible. 


Table  1 


SOME  COMPUTATIONAL  EXPERIENCE  WITH  THE  ESOD-'3  PROGRAM 


Problem 

No.  of 
Variables 

N 

j-1 

No.  of 
Const  ..aints 
(Nonredundant) 

No.  of 
Subproblems 
Required 

So lution 
Time 
(7044) 

Guaranteed 

Maximum 

Possible 

Error 

(7.) 

A1 

37 

13 

3 

18" 

1.85 

A2 

48 

15 

3 

17" 

1.33 

A3 

60 

17 

4 

23" 

1.30 

A4 

72 

20 

4 

22" 

1.93 

A5 

84 

27 

3 

23" 

1.33 

A6 

96 

30 

5 

39" 

2.56 

B1 

43 

9 

1 

4" 

1.73 

B2 

83 

23 

3 

53" 

1.89 

B3 

122 

33 

2 

49" 

6.85 

B4 

154 

39 

2 

1'45" 

6.90 

Cl 

26 

2 

2 '20" 

1,82 

C2 

71 

3 

4 '25" 

2.53 

C3 

96 

4 

5 '06" 

5.69 

C4 

381 

123 

3 

4*27" 

6.85 

CONCLUSIONS 

The  central  conclusion  from  this  experience  is  that  the  number 
of  subproblems  that  must  be  solved  remains  extremely  small,  even  when 
the  problem  becomes  quite  large.  This  remarkable  result  is  certainly 
a  stronger  justification  for  using  the  present  reduction  strategy  than 
could  reasonably  have  been  hoped  for  in  advance. 

This  experience  also  shows  that,  in  spite  of  the  efficiency- 
reducing  shortcut  mentioned  above  under  implementation,  ESOD-3  can 
solve  large  practical  problems  in  a  reasonable  time.  The  guaranteed 
maximum  possible  error  levels  attained  for  the  optimal  value  of  (P) 
were  generally  sufficient  to  assure  three-  or  four-place  accuracy  in 
the  estimated  frequencies--probably  more  than  sufficient  for  most  uses. 


-33- 


It  is  of  interest  to  conpare  these  computing  times  and 

accuracies  with  results  obtained  by  applying  Graves'  routine  directly 

to  (P)  itself  (without  using  the  reduction  procedure).  For  problem 

Al,  in  8  minutes  and  over  300  iterations  the  guaranteed  mtucimum  possible 

error  (GMPE)  was  30  percent;  in  20  minutes  and  over  1000  Iterations 

the  GMPE  was  18  percent.  For  problem  A2,  the  GMPE  was  600  percent  in 

8  minutes  and  34  percent  in  30  minutes.  The  performance  on  problems 

A3  through  A6  was  even  poorer  when  compared  with  the  results  shown  in 

* 

Table  1.  The  story  was  similar  for  Series  B  and  C.  This  reflects 
the  usual  tendency  of  first-order  gradient  methods  to  converge  slowly 
in  the  neighborhood  of  the  optimum.  That  the  very  same  gradient 
method  should  perform  so  much  better  in  terms  of  time  and  accuracy, 
when  used  in  conjunction  with  the  present  reduction  strategy,  constitutes 
a  compelling  example  of  the  power  of  this  strategy. 

h 

Some  of  the  larger  problems  could  not  even  be  attempted,  as  the 
direct  application  of  Graves'  routine  to  (P)  is  less  economical  In 
terms  of  primary  computer  storage  than  the  present  approach. 

(P)  and  (Pg)  appear  to  be  particularly  difficult  and  ill-behaved 
in  this  regard. 
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